Method and Apparatus for Associating Identifiers with Content

ABSTRACT

Watermark detection in an image or the like can be optimized by exploiting the innate biases in the image to emphasize the watermark signal. The watermark signal can be trial-located with different origins in the image to find one that yields improved results. Similarly, the image can be processed (e.g., by changing resolution, rotation, or compression) so as to change the innate biases to better reinforce the watermark signal. Compression of an image can be done in accordance with a desired identifier, with the compressor deciding which image components to retain and which to discard based, in part, on the identifier that is to be associated with the image. The techniques are also applicable to other forms of content, such as audio. A variety of other arrangements are also detailed.

RELATED APPLICATION DATA

This application is a division of application Ser. No. 11/060,975, filedFeb. 17, 2005 (now U.S. Pat. No. 7,177,443), which is a continuation ofapplication Ser. No. 10/003,717, filed Oct. 22, 2001 (now U.S. Pat. No.6,920,232), which is a continuation of application Ser. No. 09/434,757,filed Nov. 4, 1999 (now U.S. Pat. No. 6,307,949), which is acontinuation-in-part of copending application Ser. No. 09/186,962, filedNov. 5, 1998, which is a continuation of application Ser. No.08/649,419, filed May 16, 1996 (now U.S. Pat. No. 5,862,260), which is acontinuation-in-part of PCT application PCT/US96/06618, filed May 7,1996.

The present subject matter is related to that disclosed in theassignee's other patents and applications, including U.S. Pat. No.5,862,260, and application Ser. No. 09/074,034 (now U.S. Pat. No.6,449,377), Ser. No. 09/127,502 (now U.S. Pat. No. 6,345,104), Ser. No.09/164,859 (now U.S. Pat. No. 6,374,036), Ser. No. 09/292,569 (nowabandoned in favor of continuing application Ser. No. 10/379,393), Ser.No. 09/314,648 (now U.S. Pat. No. 6,681,028), Ser. No. 09/342,675 (nowU.S. Pat. No. 6,400,827), and Ser. No. 09/343,104 (now abandoned infavor of continuing application Ser. No. 10/764,430).

BACKGROUND

Watermarking is a well-developed art, with a great variety oftechniques. Generally, all vary an original signal (corresponding, e.g.,to audio or image data—video being considered a form of image data) soas to encode auxiliary data without apparent alteration of the originalsignal. Upon computer analysis, however, the auxiliary data can bediscerned and read. (For expository convenience, the followingdiscussion focuses on image data, although the same techniques aregenerally applicable across all watermarking applications.)

A problem inherent in all watermarking techniques is the effect of theunderlying image signal. In this context the underlying imagesignal—although the intended signal for human perception—acts as noisefor purposes of decoding of the watermark signal. In most cases, theenergy of the image signal far exceeds that of the watermark signal,making watermark detection an exercise in digging out a weak signalamidst a much stronger signal. If the encoded image has been degraded,e.g., by scanning/printing, or lossy compression/decompression, theprocess becomes still more difficult. As watermarks become increasinglyprevalent (e.g., for device control, such as anti-duplication featuresin reproduction systems), the importance of this problem escalates.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows how an image may be tiled with a watermark.

FIGS. 2 and 3 are flow charts illustrating methods according todifferent embodiments.

FIGS. 4 and 5 are flow charts illustrating methods according to otherembodiments.

DETAILED DESCRIPTION

To mitigate the problem of detecting a watermark signal in the presenceof a much-stronger image signal, certain choices are made early in theprocess—at the embedding operation.

The “noise” introduced by the image signal doesn't always hurt thedetection process. Sometimes innate biases in pixel values, or otherimage characteristics (e.g., DCT, wavelet, or other transformcoefficients), can actually serve to accentuate the watermark signal andthereby simplify detection.

Starting with a perhaps overly simple case, consider an image prior towatermark encoding. If the un-encoded image is analyzed for the presenceof a watermark, none should be found. However, there are cases in whichthe innate image characteristics sufficiently mimic a watermark signalthat a phantom watermark payload may nonetheless be decoded. If theapplication permits, the user may then encode the image with thiswatermark payload. This encoding just accentuates the phantom datasignal coincidentally present in the image. Even if all of the addedwatermark energy is somehow thereafter lost, the watermark may still bedetectable.

Most watermark decoding algorithms are designed to guard againstdetection of phantom watermarks in un-encoded images. For example, thealgorithms may look for checksum bits in the watermark payload; if thepayload bits don't correspond as expected to the checksum, the decodermay simply report that no watermark is detected. Other algorithms mayemploy some confidence metric for each of the decoded bits (e.g.,signal-to-noise ratio). Unless the confidence metric for all the decodedbits exceeds a threshold value, the decoder may again report that nowatermark is detected.

In applying the principles detailed in this specification, it isgenerally desirable to disable or circumvent mechanisms that guardagainst detection of phantom data so as to essentially force the decoderto make its best guess of what the watermark payload is—assuming thereis a watermark present. In the case just discussed, this would involvecircumventing checksum checks, and lowering the detection confidencethresholds until watermark data is discerned from the un-encoded image.

The approach just-discussed assumes that the image proprietor has totalfreedom in selection of the watermark payload. This may be the case whenthe image is being secretly marked with an identifier whose purpose isto identify unauthorized dissemination of the image—in such case, theidentifier can be arbitrary. More commonly, however, the watermarkpayload data cannot be so arbitrarily selected.

A variant of the foregoing considers the phantom presence of specificwatermark payload bits in the un-encoded image. Many watermark encodingtechniques essentially encode each payload bit position separately(e.g., each bit corresponds to specific image pixels or regions, or tospecific transform coefficients). In such arrangements, the un-encodedimage may mimic encoding of certain payload bits, and be indeterminate(or counter) as to others. Those bits for which the image has an innatebias may be incorporated into the watermark payload; the other bits canbe set as may befit the application. Again, the image is thenwatermarked in accordance with the thus-determined payload.

(The notion that an image may have a preference for certain watermarkpayload data is expressed in various of my earlier patents, e.g., inU.S. Pat. No. 5,862,260, as follows:

-   -   The basic idea is that a given input bump has a pre-existing        bias relative to whether one wishes to encode a ‘1’ or a ‘0’ at        its location, which to some non-trivial extent is a function of        the reading algorithms which will be employed, whose (bias)        magnitude is semi-correlated to the “hiding potential” of the        y-axis, and, fortunately, can be used advantageously as a        variable in determining what magnitude of a tweak value is        assigned to the bump in question. The concomitant basic idea is        that when a bump is already your friend (i.e. its bias relative        to its neighbors already tends towards the desired delta value),        then don't change it much. Its natural state already provides        the delta energy needed for decoding, without altering the        localized image value much, if at all. Conversely, if a bump is        initially your enemy (i.e. its bias relative to its neighbors        tends away from the delta sought to be imposed by the encoding),        then change it an exaggerated amount. This later operation tends        to reduce the excursion of this point relative to its neighbors,        making the point less visibly conspicuous (a highly localized        blurring operation), while providing additional energy        detectable when decoding. These two cases are termed “with the        grain” and “against the grain” herein.)

Again, the foregoing example assumes that the user has flexibility inselecting at least certain of the payload bits so as to exploitwatermark biases in the image itself. Commonly, however, this will notbe the case. In such cases, other approaches can be used.

One approach is to vary the origin of the encoded watermark data withinthe image. “Origin” is a concept whose precise definition depends on theparticular encoding technique used. In the watermarking techniquesdisclosed in the commonly-owned patents and applications, thewatermarking is performed on a tiled basis (FIG. 1), with a squarewatermark data block 14 (e.g., 128×128 pixels) being repetitivelyapplied across the image 12. Heretofore, the upper left hand pixel ofthe first data block is made coincident with the upper left hand pixelin the image (the latter is the origin). Thereafter, the watermark blockis tiled horizontally and vertically across the image, repeating every128 pixels. At the right and bottom edges, the tiled data block mayoverlie the edge of the image, with some of the block lost off theedges. This arrangement is shown in FIG. 1.

The assignment of the origin to the upper left hand corner of the imageis a matter of convention and simplicity more than design. The origincan be moved to the next pixel to the right, or the next pixel down,without impairing the watermark's operation. (The decoding techniquedetailed in the commonly-owned patents and applications determines thelocation of the origin by reference to a subliminal graticule signalembedded as part of the watermark. A related system is shown in U.S.Pat. No. 5,949,055. By such arrangements, the encoding origin cangenerally be placed arbitrarily.) Indeed, in the case just cited, thereare 16,384 possible origins (128*128) in the image that can be used.(Beyond the first 128×128 pixels, the tiling starts duplicating one ofthe 16,384 states.)

When an un-encoded image is decoded using the upper left hand pixel asthe origin, a first set of watermark payload biases, as described above,may be revealed. If the origin is moved a single pixel to the right, asecond set of watermark payload biases becomes evident. Likewise foreach of the 16,384 possible origins.

For short payloads (e.g., up to 12 bits), it is probable that one ormore of the phantom watermarks that may be discerned from the un-encodedimage—starting with different origin points—will exactly yield thedesired payload. For longer payloads, an origin can likely be selectedthat will exhibit a phantom bias for many of the payload bits. The taskthen becomes one of searching for the origin that yields suitableresults. (“Suitable” here depends on the application or the preferencesof the user. At one extreme it can mean finding the single origin withinthe 16,384 possible that yields the best possible phantom watermarkresults. If several origins yield the same, desired, phantom watermarkbiases, then each can be analyzed to discern the one yielding the bestsignal-to-noise ratio. In other applications, searching for a suitableorigin can mean finding the first of perhaps several origins that yieldthe desired innate payload bit biases-regardless of whether there may beothers that yield the same payload bit biases at better signal-to-noiseratios. In still other applications, a suitable origin can be any pointthat yields innate payload bit biases better than the normalupper-left-corner-pixel case. Etc.)

Except in limited circumstances (e.g., encoding a watermark in a singleimage that may be replicated billions of times, such as a banknote), anexhaustive search to find the single best origin may be socomputationally burdensome as to be impractical. There may commonly beshortcuts and clues based on particular image characteristics and theencoding/decoding algorithms that can be employed to speed the searchprocess.

The “origin” need not be a spatial location. It can be any otherreference used in the encoding process. Quantization-based watermarkencoding schemes, for example, may tailor the quantization levels inaccordance with the particular innate biases of the image to encodedesired watermark data.

In other embodiments, the suitability of an image to accept a particularwatermark having a particular origin may best be ascertained bymodifying the image slightly, and analyzing the modified image todetermine watermark suitability. For example, a trial watermark(complete or incomplete, reduced amplitude or full amplitude) might beinserted into part or all of the image with a trial origin. Theanalyzing could then include an attempted reading of the watermark toyield a performance metric (e.g., signal-to-noise ratio). Based on theresults thus achieved, the suitability of the image to host suchwatermark data with that particular origin can be assessed, and theprocess repeated, if desired, with a different origin. After thuscharacterizing the suitability of the image to accept watermarks withdifferent origins, the image may be watermarked using the origin foundto yield the best performance.

Although the foregoing discussion focused on changing the origin of thewatermarking, other parameters can also be varied to effect the “match”between the innate image characteristics and the watermark data. Onesuch parameter is image resolution. Another is image rotation. Yetanother is compression.

Consider a vector graphic image that is “ripped” to yield a set of pixeldata. The conversion can yield any desired pixel spacing (resolution),e.g., 600 dpi, 720 dpi, etc. The different resolutions will yield imagesthat may be differently suited to host a particular set of watermarkdata. By analyzing the image at different resolutions, one may be foundthat provides innate image attributes that best tend to reinforce thedesired watermark signal.

Similarly, with rotation. It is not essential that the image be encodedwith the “top” oriented vertically. By rotating the image 90, 180, 270degrees (or even to intermediate rotation states) prior to watermarkencoding, a state may be found that provides image attributes tending toassist with the watermark encoding.

In still other applications, image attributes may be changed bycorrupting the image through differing degrees of lossycompression/compression. To human observers, the results of differentcompression processes may be imperceptible, yet in the encoding domains,the resulting changes may make a particular image better- orworse-suited to encoding with a particular watermark. Again, varioussuch modifications can be made to the original image to try and find acounterpart image that coincidentally has attributes that tend toreinforce the desired watermark signal.

Image modifications other than changing resolution, rotation, andcompression can similarly be pursued; these three are exemplary only.

Reference was sometimes made above to image attributes that“coincidentally” tended to reinforce the desired watermarking signal. Inparticular cases, such attributes needn't always be left to chance. Forexample, in the compression-based approach just-discussed, compressionalgorithms have a great deal of flexibility in determining what imagecomponents to maintain, and which to omit as visually superfluous. Thedecision whether or not to omit certain image components can be madedependent, in part, on a priori knowledge of a watermark payload that isto be encoded (or retained) in the image, so as to optimize the innatebiases in the decompressed image accordingly. Indeed, the entirewatermark encoding process may be realized through a suitablecompression algorithm that operates to retain or discard imageinformation based at least in part on the watermark-related attributesof the resulting image after processing. Such arrangements are depicted,e.g., in FIGS. 4 and 5.

In still other embodiments, a multi-way optimization process may beperformed. The original image can be analyzed to find which of severaldifferent origins yields the best results. The original image can thenbe modified (e.g., resolution, rotation, compression), and a variety ofdifferent origins again tried. Still further modifications can then bemade, and the process repeated—all with a view to optimizing the image'sinnate suitability to convey a particular watermark.

As is familiar to those skilled in the arts, the foregoing methods maybe performed using dedicated hardware, through use of a processorprogrammed in accordance with firmware or software, etc. In the lattercase the processor may include a CPU and associated memory, togetherwith appropriate input and output devices/facilities. The software canbe resident on a physical storage medium such as a disk, and can beloaded into the processor's memory for execution. The software includesinstructions causing the CPU to perform the analysis, search,evaluation, modification, and other processes detailed above.

The variety of watermarking techniques is vast; the technology detailedabove is believed applicable to all. The variety of watermarkingtechniques is illustrated, e.g., by earlier cited patents/applications,and U.S. Pat. Nos. 5,930,469, 5,825,892, 5,875,249, 5,933,798,5,916,414, 5,905,800, 5,905,819, and 5,915,027.

Having described and illustrated the principles of my work withreference to various embodiments thereof, it will be recognized thatthis technology can be modified in arrangement and detail withoutdeparting from such principles. For example, while the detailedembodiment particularly considered image data, the same principles areapplicable to audio data. (The “origin”—based approaches would commonlyuse a temporal origin.) Similarly, the detailed techniques are notlimited solely to use with digital watermarks in a narrow sense, butencompass other methods for processing an image to encode otherinformation (e.g., for authentication or digital signature purposes, forimage-within-an-image encoding, etc.—all regarded as within the scope ofthe term “watermark” as used herein.) Accordingly, I claim all suchembodiments as may come within the scope and spirit of the followingclaims and equivalents thereto.

1. A method of steganographically encoding digital content with adigital watermark, comprising: receiving data representing digitalcontent; selecting a digital watermark pattern to be conveyed by saidcontent data, said selection considering innate characteristics of thecontent data tending to mimic a particular watermark pattern; andaltering said content data to further emphasize said selected digitalwatermark pattern.
 2. In a method of steganographically encoding digitalcontent with a digital watermark, the method including determining adigital watermark pattern to be conveyed by said content data, andaltering the content data to convey said digital watermark pattern, animprovement comprising applying a watermark extracting process to thedigital content prior to determining the digital watermark pattern to beconveyed, and using information resulting from said process in selectinga particular digital watermark pattern.
 3. In a method of watermarkingcontent data by processing same in accordance with a watermark signal,an improvement comprising selecting said watermark signal by referenceto said content data.